1. Field of the Invention
The present invention relates to a method and system for detecting the relative position of a pair of images which is suitable for an automatic focusing device of a camera, and more particularly to a method and system for detecting the relative position of the two images on the basis of a pair of image data indicative of light intensity distribution of the images from a pair of optical sensor arrays that receive the images through light paths separated from each other in space.
2. Description of Prior Art
In a method for automatic focusing of a camera, a pair of optical sensor arrays receives the image of a subject through light paths separated from each other in space. The relative position of the images or the offset of the images from a reference point on the pair of sensor arrays is detected. Focusing of the camera lens is effected by using the relative position as a measure representing the distance and direction to the subject and an amount of the offset of the focal point of the lens. FIG. 3 to FIG. 5 show applied examples of this method for detecting the relative position of a pair of images.
FIG. 3 shows diagrammatically an example of detecting a distance d to a subject 1. A pair of small lenses 2L,2R, spaced apart by a distance b from each other, is mounted on a camera. The light from the subject 1 is received at the small lenses 2L,2R through different light paths 3L,3R. A pair of optical sensor arrays 10L,10R is placed at the focal points of the small lenses 2L,2R to form images IL,IR, respectively, thereon. Since the subject 1 is framed by a view finder so that the subject 1 is in front of the small lens 2L on the left, the image IL of the subject 1 is positioned at the same location on the optical sensor array 10L. The image IR, as apparent from the figure, occupies different locations on the optical sensor array 10R on the right in accordance with the distance d to the subject. In other words, the image IR on the optical sensor array 10R is offset from a reference position R at which the image is formed when the subject 1 is at an infinite distance. Thus the distance d can be found from a simple trigonometric formula, d=b .multidot.f/S where S is the amount of the offset of the image relative to the reference position R. Even so, the distance d is not actually calculated for focusing the camera, but rather the amount of offset S is directly used for focusing the lens.
FIG. 4 shows another example of detecting a distance where the subject 1 is not located in front of either the left small lens or the right small lens but is in front of an intermediate position between the two small lenses 2L,2R, for example. In this case, both of the images IL,IR on the two optical sensor arrays 10L,10R are, in transverse direction, offset from the reference position R at which an infinitely distant subject causes an image. It should easily be noted that the distance d can be determined by the aforementioned equation, d=b .multidot.f/S using the sum, S=SL+SR, of the offsets of the respective images IL and IR. It should also be noted that detecting the distance merely requires detecting the relative position between the images IL,IR or the offset of the images IL,IR on the optical sensor arrays 10L,10R from their reference positions.
FIG. 5 shows an example of detecting an out-of-focus condition in a single lens reflex camera, for example. A mirror 5 is placed between a taking lens 4 and a film 6. The light from the subject 1 reflected by the mirror 5 forms images through a pair of small lenses 2L,2R on the horizontally positioned sensor array portions 10L,10R of the optical sensor array 10. Although the images on these horizontally positioned light sensor array portions 10L,10R are formed through the taking lens 4, the optical sensor array portions 10L,10R receive the images through different portions of the lens 4 and separate light paths 3L,3R in space.
On the right of FIG. 5 are shown three cases of image locations on the optical sensor array depending on the focus of the lens 4. I shows the images when the lens 4 is in-focus. As apparent from the figures, the peak portions of the image If move closer to each other when the taking lens 4 is in front-focus condition, and the peak portions of the image Ib move away from each other when the taking lens is in back-focus condition. Thus, the amount of out-of-focus and the direction thereof, i.e., front or back, can be found by detecting the actual offsets SL,SR of the image peaks or the sum S of these offsets from the reference position R at which the image is in-focus. Detecting the amount of out-of-focus is, therefore, nothing but detecting the relative position of the images on the two light sensor arrays 10L,10R.
FIG. 6 shows, by way of example, a method for finding the relative position of the images on the optical sensor array. The left and right image data DL,DR are each, by way of example, a collection of m data indicative of light intensity distribution of the images IL,IR. Images IL,IR are shown in a very simple shape as the images on the optical sensor arrays 10L,10R. In this example, the optical sensor arrays 10L,10R have the same number of light sensors as those in FIG. 4 and FIG. 5. Partial image data PL,PR consisting of n data, respectively, are sampled from the two image data DL,DR as verification data. The correlation between the verification data is examined in order to detect the the relative position of the image data DL,DR. Of course, m is greater than n, so there are a plurality of ways of sampling the verification data PL,PR from the image data DL,DR as shown in FIG. 6. Different combinations of these verification data sampled are defined as Ck (where k=1 to P) as shown. When sampling, both verification data consist of successive data on the optical sensor arrays. In successive combinations one of the verification data is shifted by one data alternately with the other verification data as shown. Since the number of data of the respective image data DL,DR is m and the number of data of the respective verification data PL,PR is n, there are m-n different ways of shifting. Therefore, there are 2(m-n)+1=P different combinations, Ck, of the verification data. An evaluation value F indicates the level of correlation of the respective verification data PL,PR with respect to the combinations Ck. The following equation, for example, is used as an evaluation function for calculating the evaluation value F. ##EQU1## where i and j are variables for specifying each data within the verification data PL,PR. For example, i and j vary from i=m-n+1 to m and j=1 to n at the first combination C1, to i=1 to n and j=m-n+1 to m at the final combination CP. The respective data Li,Rj are each, for example, 8-bit digital data. Where the above-mentioned evaluation function is used, the evaluation value is small when the verification data are of high correlation, and large when the verification data are of low correlation. This is apparent from the fact that the evaluation value F is equal to zero when n data of Li and Rj, respectively, of both verification data PL,PR are all coincident.
FIG. 7 shows an example in which the evaluation value F is in the abscissa and the variable k indicative of combinations is in the ordinate. The shape or the envelope of the plotted evaluation of F is, of course, dependent upon the shape of light intensity distribution of the images IL,IR and may be very uneven as shown. In this example, a low evaluation value indicates a high correlation. The lowest value implies that a position at which the n verification data PL,PR, i.e., the image portions corresponding to the n verification data PL,PR in the vicinity thereof, are almost completely coincident with each other has been found. Thus, taking the combination km having a minimum value, km actually indicates the relative position of the images on the two sensor arrays 10L,10R. The offset S between the images can be determined from the difference between the combination number km and the combination number kr corresponding to the reference position in the figure.
The offset S or the combination number km corresponding to the highest correlation thus obtained can be applied directly to focusing of the taking lens of a camera, for example. The combination number km and offsets are integers, but can be converted into fractional numbers through interpolation on the basis of the shape or the envelope of the plotted evaluation value F near the combination number km. In addition, if the optical sensor arrays 10L,10R each have a different number of light sensors, one fixed verification data consisting of n data is chosen, for example, from the image data DL from the left optical sensor array 10L that is assumed to have the lesser number of light sensors. The correlation evaluation value F is then calculated for combinations of the above verification data DL and m-n+1 verification data sampled from the image data DR, which consists of n data corresponding to the optical sensor array 10R on the right. The combination number km indicative of the highest correlation may be determined in exactly the same way as that mentioned above.
As apparent from the previous description, with a conventional method for detecting the relative position of a pair of images, a plurality of verification data are sampled from at least one of the image data of the subject on the left and right light sensor arrays, and then the combination that shows the highest correlation of all combinations of the verification data on the left and right is selected. In many cases, the combination of the highest correlation can be determined on the basis of the evaluation values F or a shape or the envelope that the plotted evaluation value F forms with respect to the combination number. The problem is that sometimes the combination having the highest correlation may not be correctly determined due to an abnormal shape or the envelope of the evaluation value F. This problem is often present where the subject is rather simple, which is the case when the images IL,IR are of a very simple shape and changes within the image are concentrated on one local portion with no significant changes in the rest of the image, as for example, a sight through a camera finder where one electric power pole is standing in the background of a blue sky or a monotonic wall.
Where there is not much change in the background of an image, if verification data including data corresponding only to this background is sampled by chance from the left and right image data, some data may substantially coincide with each other, resulting in a combination of a false highest correlation. FIG. 7 shows this situation at the left and right ends thereof, in which the evaluation value F is of a very low value for large combination numbers k at the right end, while for small combination numbers the evaluation value F is slightly larger than that for the highest correlation point km but has a local minimum value. In this manner, if there is not much change in the background of the subject, then a false highest correlation point of the same evaluation value as or very close to the true highest correlation point may sometimes appear. Thus, the conventional method may have some difficulty in detecting the true relative position of the image pair or cannot detect it at all.